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ABSTRACT 

Previous studies have documented the effectiveness of 
feedback in improving students 1 achievement, but few studies have 
examined the effectiveness of summaries or reviews. This study 
examines the respective and combined effects of feedback and review 
on students 1 achievement, retention, and level of cognitive 
development. A total of 55 ninth-grade students comprised five 
treatment groups which received an interactive slide/sound computer 
earth science lesson on "The History of the Earth." All groups 
received 26 pelf-test questions throughout the program. The control 
group received no feedback to their responses to the self-test 
questions and no reviews for each of the seven sections of the 
program. One group received non-content feedback for the responses 
and no reviews, a second group received content feedback and no 
reviews, a third group received non-content feedback and reviews; and 
the fourth group received content feedback and reviews. All subjects 
were given a 28-item achievement test immediately following the 
computer treatment and the same achievement test one week later. The 
study concluded that providing students with feedback produced 
significant increases in achievement, retention, and higher-order 
learning while a review produced significant improvement only in 
higher-order learning. (Author/CW) 
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ABSTRACT 



Previous studies have documented the effectiveness of feedback in improving 
students* achievement, but few studies have examined the effectiveness of summaries or 
reviews. This study examines the respective and combined effects of feedback and 
review on students achievement, retention, and level of cognitive development. 

A total of 55 ninth-grade students comprised five treatment groups which received 
an interactive slide/sound computer earth science lesson on The History of the Earth. All 
groups received 26 self-test questions throughout the program. The control group 
received no feedback to their responses to the self-test questions, and no reviews for 
each of the seven sections of the program. One group received non-content feedback for 
the responses and no reviews, a second group received content feedback and no 
reviews, a third group received non-content feedback and reviews, and the fourth group 
recieved content feedback and reviews. All subjects were given a 28 item achievement 
test immediately following the computer treatment and the same achievement test one 
week lacer. 

Results indicated that 

1. R* the main effect, content feedback produced better results than reviews 
and reviews produced better results than no feedback or reviews on 
students achievement and retention. (Combined content feedback and 
reviews produced highest, but not significantly different, means.) 

2. Achievement and retention difference were attributable to the higher 
cognitive level questions; no treatment effects resulted from low cognitive 
level items. Higher level self-test questions provided a predictability for 
retention. 

3. Although the amount of informational feedback increased the students* 
program time proportionately, productivity was significantly greatest for 
content feedback. 

The study concluded that providing students with feedback produced significant 
increases in achievement, retention, and higher-order learning while a review produced 
significant improvement only in higher-order learning. 

The study suggests interesting implications for the efficiency of the interactive form 
of science instruction and the impact of higher cognitive level feedback on retention. 
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THE EFFECTS OF SLIDE/SOUND COMPUTER-BASED INSTRUCTION 
ON STUDENTS' ACHIEVEMENT AND RETENTION 

Purpose 

The importance of feedback in classroom situations has been firmly 
established in the literature. According to Briggs (1977), it has been shown that 
learning from print materials can be enhanced by providing "self-test" 
questions which learners respond to before taking a different test over the 
objectives of the lesson. Reinforcement of learning should be immediate and 
systemitized, and this should result in more effective learning (Chambers and 
Sprecher, 1974). Research by Ellis, Kinoske, Wolfect, and Montague (1982), 
Surber and Anderson (1975), Morgan (1961), and Gaynor (1981), to name a 
few, have all indicated the importance of feedback in enhancing learning. 

Feedback is used, in a generic sense, to describe any of the numerous 
procedures that are used to tell a learner if an instructional response is right or 
wrong. It ranges along a continuum from the simpliest yes-no format to the 
presentation of substantial or corrective remedial information that may extend 
the response context or even add new material to it (Kulhavey, 1977). 

Review, more commonly given when a tutorial is about to end 
permanently, provides summary statements about the information in the lesson. 
A summary might be a list of major points or a paragraph summing up the 
purpose of the lesson (Alessi and Trollip, 1985). Gaynor (1981), indicates that 
end of session feedback, in essence a review, also facilitates learning more 
abstract and conceptual material in which the learner must apply higher-order 
thinking skills. Lasuy, end of session feedback seems to facilitate long-term 
retention, especially with high mastery students. 

Will the effects ^f feedback and/or review enhance students' learning and 
retention when used in a computer program? The purpose of this study is to 
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examine the effects on learning of varying feedback and review in an interactive 
slide/sound computer earth science lesson on "The History of the Earth." 
Procedures 

The hardware utilized in this study consisted of an Apple He computer with 
a color monitor and two disk drives and a Bell and Howell Ringmaster 
slide/sound projector. The slide/sound projector, which is interfaced with the 
computer, has the capability of projecting a series of 2" x 2" slides from a tray 
onto a screen located on the front of the projector while at the same time playing 
a cassette tape containing an audio script describing each slide. 

The interactive earth science lesson consisted of 78 color slides depicting 
various plants and animals and illustrations of the appearance of the North 
American continent during each geologic period of the earth's history. The 
central theme of the lesson was the development of the earth and its living 
organisms through time. The audio script on the cassette tape provided students 
with a running commentary on how the earth was developing by discussing the 
scenes or illustrations on each slide. The program was divided into seven 
sections and at the end of each section students were asked, on the computer, a 
number of multiple choice self-help questions concerning the material in the 
section. There were 26 of these self-help questions in the program. After 
answering each question, students were then given non-content (knowledge of 
results) feedback or content (informational) feedback, by the computer, in 
response to their answers. Some students were also given a review at the end 
of each section. The slides, audio description, questions, feedback and review 
were scrutinized by four experts in the field and found to have a high content 
validity. Using the Kuder-Richardson 20 formula, the reliability of the 
self-help questions was found to be 0.7104. 
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After completion of the earth science lesson, students took a paper and 
pencil achievement test consisting of 28 multiple choice questions similiar to the 
self-help questions. These items determined the students* understanding of the 
development of the earth throughout its history as discussed in the computer 
lesson. Of these 28 achievement items, 8 were judged to be lower-order 
questions (knowledge and comprehension level) while 20 items were judged to 
be higher-order questions (application level and above). Using the 
Kuder-Richardson 20 formula, the reliability of these achievement questions 
was found to be 0.7130. 

The population consisted of 55 ninth-grade earth science students from one 
high school located in a rural Pennsylvania school district. An earlier pilot 
study was conducted at a neighboring high school in the same school district to 
establish the reliability of the self-help questions and the achievement instrument 
used in the study. There were 44 ninth-grade earth science students that 
participated in this pilot study. 

In the reported study, the 55 students were randomly assigned to one of 
five treatment groups as shown below: 
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Figure 1. Research Design of Earth Science Lesson 

One group (Cell I) received non-content feedback (NCF) which told the 
students only if their responses to the self-help questions were correct or 
incorrect. They also received no review (NR) at the end of each section. One 
group (Cell II) recieved content feedback (CF) which told them why their 
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answer was correct or incorrect and also received no review at the end of each 
section. One group (Cell III) received non-content feedback but had a 
review (R) at the end of each section. The final group (Cell IV) had content 
feedback and a review. A control group (Cell 0) received no feedback except 
"Thank You For Your Answer" and had no review at the end of each section. 

Arrangements were made for each student to leave his/her earth science 
class and come to a science trailer adjacent to the school. There they 
individually ran through the earth science computer lesson, viewing the slides 
and listening to the description, answering the self-help questions, and 
receiving feedback and/or review according to the treatment group they were 
placed in. The program required an average of about 45 minutes to complete. 
Upon completion of the computer lesson, each student moved to a different part 
of the trailer where he/she took the achievement test which took about 15 
minutes to finish. Because of total cooperation of the administration and 
faculty, the students had no problem leaving class or returning to class after 
completing the lesson and test. 

After a one week delay, the same achievement test was administered to the 
participants as a retention test. This test was given in the library so the 
participating students left their earth science class and were given the test all at 
once. As before, the test took approximately 15 minutes to complete after 
which all students returned to their earth science class. 
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Results 

Means and standard deviations for each of the five cells, according to 

low-level, high-level, and total questions correctly answered, follow in Table 1. 

Table 1 

Low-Level, High-Level, Total Mean Scores and Standard Deviations 
for Achievement and Retention Tests by Treatment 

Achievement Test Retention Test 





T T 

LL 


TJT 


Total 


LL 


HL 


Total 




(n=8) 


(n=20) 




(n=28) 


(n=8) 


(n=20) 


(n=28) 


NF-NR 


4.78 


6.49 




11.27 


3.91 


6.91 


10.82 


(control) 


(0.90) 


(1.86) 


(2.24) 


(0.94) 


(1.64) 


(3.76) 


NCF-NR 


5.18 


8.73 


13.91 


4.18 


7.37 


11.55 




(1.25) 


(3.74) 


(4.09) 


(0.98) 


(2.42) 


(2.70) 


CF-NR 


5.64 


12.91 


18.55 


4.18 


11.27 


15.45 




(1.03) 


(3.62) 


(3.98) 


(0.98) 


(3.23) 


(3.19) 


NCF-R 


4.72 


11.64 


13.36 


4.09 


10.73 


14.82 




(1.35) 


(4.25) 


(4.52) 


(0.70) 


(3.90) 


(4.24) 


CF-R 


5.27 


13.82 


19.09 


4.09 


12.64 


16.73 




(1.01) 


(3.74) 


(3.51) 


(0.30) 


(3.56) 


(3.69) 


Mean 


5.12 


10.72 


15.84 


4.09 


9.78 


13.87 




(1.11) 


(3.44) 


(3.83) 


(0.78) 


(2.95) 


(1.87) 



As can be seen from table 1 above, the group that received content feedback 
and had reviews scored better than the other groups in both acheivement and 
retention test scores, while the group that received content feedback but had no 
reviews scored slightly lower but better than the other groups. 
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The achievement test scores and retention test scores were analyzed using a 

multiple analysis of variance (MANOVA) program found on a main-frame 

SPSS statistical package. These results appear in Tables 2 and 3, 

Table 2 

Summary of MANOVA Results of 
Achievement Test Scores 

Tests of Significance for Main Effects Using Sequential Sums of Squares 



SOURCE DF MS F P 



Review 1 590.945 1.194 0.281 

Feedback 1 3797.195 7.670 0.008* 

Review/Feedback 1 268.768 0.542 0.466 

Error 40 495.057 



Total 43 
*F 01 «7.31 

The results from Table 2 indicate that content feedback performed better than 
non-content feedback on achievement test scores. However, there was no 
significant difference between review and no review performances and no 
significant interaction. 

Table 3 

Summary of MANOVA Results for 
Achievement versus Retention Test Scores 



Tests of Significance for Retention Using Sequential Sums of Squares 



SOURCE 


DF 


MS 


F 


P 


Retention 


1 


4177.876 


124.946 


<0.001** 


Review x Retention 


1 


92.365 


2.726 


0.104 


Feedback x Retention 


1 


140.945 


4.215 


0.047* 


Rev x Feedbk x Ret 


1 


2.876 


0.086 


0.711 


Error 


40 


33.438 






Total 


44 









*F os -4.08 •♦F.001-M.6 
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The MANOVA results from Table 3 indicate that content feedback performed 
better than non-content feedback on retention test scores. Review versus no 
review test scores were not significantly different. There was a very significant 
difference between the achievement test results and the retention test results 
which indicates that test scores dropped about the same for all treatment groups 
between achievement and retention tests. 

An interesting development occurs when type of learning (low-level versus 
high-level) is analyzed. 

Tabic 4 

Summary of MANOVA Results fo; Low-Level 
versus High-Level Types of Learning 

Tests of Significance for Types of Learning Using Sequential Sums of Squares 



SOURCE DF MS F P 



Type of Learning 1 294.070 0.799 0.377 

Review x Type 1 2028.445 5.5)1 0.024* 

Feedback x Type 1 1672.195 4.543 0.039* 

Rev x Fccdbk x Type 1 334.126 0.908 0.346 

Error 40 368.068 



Total 44 
*F os = 4.08 

Table 4 indicates that feedback and review both affect types of learning on 
achievement test scores. When averaging the total means for feedback on 
achievement tests, it was found that there was a gain of only 0.25 low-level 
questions correctly answered between non-content feedback and content 
feedback treatments but a dramatic gain of 3.05 high-level questions correctly 
answered between non-content and content feedback treatments. When the total 
means for review were averaged, there was a difference of only 0.26 low-level 
questions correctly answered between no review and review treatments but a 
dramatic increase of 2.14 questions correctly answered between no review and 
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review. These results indicate that both feedback and review will increase 
achievement in higher-order learning Dut neither feedback or review 
significantly increase lower-order learning. 

The MANOVA analysis also indicated a significant difference in retention 
versus type of learning as shown in the following table: 

Tabic 5 

Summary of MANOVA Results for 
Type of Learning versus Retention 

Tests of Significance for Type of Learning vs Retention using Sequential Sums of Squares 



SOURCE 


DF 


MS 


F 


P 


Retention x Type 


1 


572.736 


9.304 


0.004* 


Review x Ret x Type 


1 


12.820 


0.028 


0.651 


Fccdbkx Ret x Type 


1 


78.445 


1.274 


0.266 


Error 


40 


61.536 







Total 43 



*F 01 =7.31 

When averaging the total means for type of learning versus retention, it was 
found that lower-level learning dropped by 1.36 questions correctly answered 
out of 8 total questions between the immediate achievement test and the 
retention test. It was also found that higher-level learning dropped by 1.28 
questions correctly answered out of 20 total questions between the immediate 
achievement test and the retention test. These results indicate that students will 
retain higher-level learning much better than they will retain lower-level 
learning. 
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An interesting comparsion between the average time of completion of the 
computer program versus the average achievement sccacs can be made. 

The following table indie .;ed the time for completion of the computer 
program for each group or cell. 

Tabic 6 

Means and Standard Deviations for 
Time of Completion of Computer Program 
Across All Treatments a 





NCF 


CF 


! 


NR 


44.09 
(2.70) 


45.55 
(2.30) 


44.82 
(2.50) 


R 


45.91 
(2.43) 


50.82 
(2.23) 


48.36 
(2.42) 




45.00 
(2.57) 


48.18 
(2.27) 


46.59 
(2.42) 



a Timcs given in Minutes 

An interesting method for analyzing the effectiveness of cwh treatment was 
invented and is referred to as the productivity . The productivity of each cell 
might be defined as the mean time or completion of the computer program 
divided by the average cell score. This relationship of time/score is referred to 
as the number of minutes needed for each point or minute§/point. The 
following table lists the cell discription and the resulting productivity in 
minutes/point for both the immediate achievement test scores and retention iest 
scores. 
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Table 7 



Mean Productivity by Treatment 
in Minutes/Point 



Achievement Test 



Retention Test 



NCF-NR 
CF-NR 
NCF-R 
CF-R 



3.17 
2.46 
2.81 
2.66 



3.82 
2.94 
3.10 
3.04 



As can be seen from Table 7, The CF - NR (cell II) has the lowest ratio of 
minutes/point and can be said to have the highest productivity. The next cell is 
CF - R (cell IV) which has the second lowest ratio of minutes/point and can be 
said to have the second highest productivity. This interesting relationship 
suggests that not only does feedback improve scores, it also provides the most 
efficient use of time. 
Conclusions 

One of the most important findings of the study is that feedback, especially 
content feedback, is of prime importance in increasing student achievement. 
These results should be expected; the more detailed feedback students receive, 
the better they should do on a similiar achievement test. As Gallini (1983) 
indicates, good feedback is essential for the cognitive development of students. 
It is interesting to note that since reviews are not significiant in increasing 
students' achievement feedback is the more important of the two treatments for 
improvement of achievement. 

A second important effect the study pointed out is that feedback is also more 
important in increasing student retention than reviews. 

A third, and one of the most important findings of the study was the effect 
feedback and review had on type of learning and the effect type of learning had 
on retention. When low-level questions were involved, neither feedback nor 
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review significantly affected achievement or retention. There was, however, a 
significant and important difference between feedback and high-level learning 
and review and high-level learning. These results dramatically point out the fact 
that both feedback and review are important in eliciting higher-level learning. 
Howe and Durr (1982) found that raising the cognitive level of students helped 
them in answering higher-level questions and they also stated that feedback 
helped raise this cognitive level in students. The analysis also pointed out the 
fact that type of learning affected retention. As expected, once students learn 
high-level concepts, skills, or theories, they are much more likely to retain both 
lower and higher-level information than if they learn only low-level factual 
information. 

In summary, in the development of future classroom lessons or CBI 
software, teachers and programmers would be well advised to include some 
form of higher-level content feedback to insure better achievement, better 
retention, and better lear ; ng of higher-level concepts for their learners. 

Ronald F. Pauline 

Assistant Professor of Education 

SUNY Geneseo 

Geneseo,NY 14454 

and 

Paul E.Bell 

Associate Professor of Education 
The Pennsylvania State University 
University Park, PA 16802 
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